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Abstract: Artificial intelligence has a wide range of applications and plays a huge role in the field of 
art as well. In the application of art painting generation, there are more and more works appearing, 
and art painting creators have difficulty in getting up to speed quickly in the face of the huge variety 
of techniques of artificial intelligence. In contrast to the traditional forms of art painting, good use of 
artificial intelligence techniques will improve the speed and quality of creators' creation. The article 
outlines the basic classification of AI, focuses on the generation of art paintings by AI in real-time 
dynamics, studies the current popular AI image techniques, analyzes the style transfer algorithm using 
neural networks, and also verifies the effect of real-time dynamic generation of art paintings. The 
comparison of the six techniques reveals that AI techniques are widely used in art painting, and fast 
style transfer is effective in real-time dynamic art painting generation. The creative experiments range 
from line drawing generation to real-time dynamic art painting image generation, showing a new idea 
of cross-border fusion creation. 


Keywords: Artificial intelligence, Painting generation, Cross-border integration, Real-time dynamics, 
Style transfer. 


1. Introduction 


Artificial intelligence has become a hot topic in modern society, covering all aspects of life and 
work. It also has infinite potential in artistic creation. Art painting is not limited to the traditional 
way of creation. Art creation needs to be based on the times and society, combined with the trend 
of the times and the needs of society. Art creation has different ways in different times. Art can be 
closer to the people by combining the characteristics of the times. In recent years, the art of artificial 
intelligence creation has attracted extensive attention in the society. Microsoft Xiaobing has not only 
held a painting exhibition, but also published a collection of poems and a single, which has been 
recognized by many people and is evolving into an excellent art creator[1]. In essence, the audience 
and creator of art are human beings. Artificial intelligence only learns and extracts the characteristics 
of human works from complex and diverse data to imitate, and does not have the soul of human 
beings. Let the computer complete the simple artistic creation and liberate the creator from the 
repeated simple work, so that he can have more energy and time to engage in artistic research. As the 
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cutting-edge technology of the times, artificial intelligence can not independently create profound 
works of art, but it can learn and imitate the creative techniques of art masters in human history, 
integrate the strengths of predecessors and imitate the grandeur of predecessors [2]. The application of 
artificial intelligence in art is developing rapidly. 

Artistic creation is an activity to express people's soul and spirit. At present, artificial intelligence 
does not have the soul and wisdom that human beings can understand [3]. The significance of 
artificial intelligence lies in serving people, like stone axes in the stone age, paper in the feudal 
age, steam engines in the steam age and light bulbs in the electrical age. It is a very important tool 
for the development and progress of human society. Artificial intelligence is divided into weak 
artificial intelligence and strong artificial intelligence. Weak artificial intelligence has no independent 
consciousness and can not actively infer and think. It only helps people get rid of simple repeated 
calculation, while strong artificial intelligence not only has the fast computing ability of computer, but 
also has the same logical reasoning ability as human beings, can solve problems independently, and 
even has the self-consciousness beyond human beings, Involving or surpassing the realm of human 
spirit and soul. The arrival time of strong artificial intelligence is far away, and it is still unknown 
whether it will appear in the development history of human civilization. Weak artificial intelligence 
has become a tool that modern people cannot leave. In all aspects of clothing, food, housing and 
transportation, artificial intelligence is helping to build a more convenient and efficient social life[4]. 
Artificial intelligence is not a tool used by only a small number of people in laboratories or research 
institutes. Everyone can use artificial intelligence to improve their professional work efficiency in 
different fields, industries, scenes and affairs[5].It provides a reference for the exploration of cross- 
border art of artificial intelligence and the creation of artificial intelligence. 

Contribution of this study: 

1) How the technology used by artificial intelligence serves art creators and improves the 
professional level of art creation is the cornerstone of artificial intelligence art painting. The goal is to 
make the creative methods of art painting more diversified and convenient, which has more room to 
play compared with the traditional creative methods. 

2) Real time dynamic art painting technology can turn the real-time image into dynamic art 
painting, upgrade the traditional art painting form to a new time and space level, give art painting 
different aesthetic impressions, and improve the creative ideas of artists. 

3) The actual verification of line drawing and style transfer technology to create real-time dynamic 
art and painting works, the training model is constructed based on the improved convolution neural 
network, and the optimization algorithm is used to improve the fluency of real-time generation, which 
is better than the efficiency of traditional painting creation. 


2. The relationship between AI and art painting 


Artificial intelligence and art painting belong to two different disciplines. The basis of cross- 
border integration is to understand the relationship between them. Let artists understand the basic 
concepts of artificial intelligence and broaden creative ideas in combination with the basic knowledge 


of art painting. Starting from the concept and classification of artificial intelligence, this paper studies 
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the role of artificial intelligence in art painting. Artists combine the differences between artificial 
intelligence and traditional creation, analyze the existing cases of artificial intelligence art painting, 


and explore the feasible way of cross-border combination. 
2.1 AI and art painting 


Artificial intelligence is the general name of a kind of program of perception, reasoning, action 
and adaptation, covering a wide range. The essential meaning of its application is to simulate the 
process of human consciousness and thinking. As a branch of computer science, it is to build a 
complex algorithm system based on computer and mathematics[6].Machine learning is a branch of 
artificial intelligence. Its essence is to design algorithms that enable computers to continuously "learn" 
and improve according to data, and predict according to existing data. Deep learning is a branch of 
machine learning. Its essence is to use multilayer neural network to simulate the neurons of human 
brain, and reproduce the way of human brain receiving and transmitting information through a large 
number of network construction, so as to achieve enough intelligence. Deep learning is the most 
widely used series of algorithms in artificial intelligence technology, mainly including convolution 
neural network, deep confidence network,cyclic neural network,short-term and long-term memory 
network, etc[7].Deep learning needs a large amount of data to train a model, which contains a 
large number of parameters of neural network. After receiving new data,it can carry out predictive 
reasoning and generation tasks. 

The degree of specialization of artificial intelligence can be reflected in many fields. More and 
more people trust artificial intelligence and make their own creation according to the decision-making 
reference and instructions provided by artificial intelligence. However,this premise is that creators 
need to understand the cross domain basic knowledge combined with artificial intelligence.Otherwise, 
artificial intelligence cannot cooperate with creators well.In terms of creative techniques, the ability 
of artificial intelligence painting is no less than that of ordinary human artists, and skilled techniques 
are difficult to distinguish from human artists[8].In the creation of traditional art painting, the artist 
needs a long cycle. The early observation and research, obtaining inspiration, carefully polishing the 
creative process, and finally obtaining a well-known work. The emergence of artificial intelligence 
painting art has certain inspiration for human artists. When facing some simple and high repetition 
rate creations, they can use artificial intelligence technology to complete them together, which can 
effectively improve their own creation speed, so as to have more time to think about the connotation 


and significance of works and create better art works. 
2.2 Application of AI in art painting 


Artificial intelligence art painting has many excellent works and creative contents. Google's 
"deep dream" art painting can transform ordinary photographic photos into images of special pictures. 
This kind of image has corresponding picture overlap and special light and shadow effect. Its principle 
is to accumulate the output of each network layer in the deep learning network. Each network layer 


can extract the characteristics of input pictures, including pixels, color sets, edge and corner features, 
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etc. in continuous iteration, These features become more and more obvious. At the same time, neurons 
can detect which pictures are in line with reality and which are not in line with reality. In this way, 
neural network can force to generate some picture scenes similar to dreams and hallucinations, similar 
to the double portrait paintings made by Renaissance painter Hans Holbein[9].The painting effect 
produced by the deep dream is like the picture seen in the dream is reproduced by the machine. It has 
illusory patterns without losing the basic recognition. "Deep dream" brings the artist's imagination to 
a new world[10]. 

In addition to using computer image generation, intelligent 3D printers can perfectly reproduce 
all kinds of famous paintings. The custom prototypes team in the Netherlands used a high-precision 
scanner to scan Van Gogh's famous paintings "starry sky", "sunflower", "wheat field", etc. the 3D 
printer was used to reproduce the paint thickness of each brush in the painting, and the subtle changes 
were engraved in depth. Then a special person was assigned to micro treat the surface to keep it highly 
similar to the original painting. In addition, expect reproducing the painting, artificial intelligence is 
also used to convert the 2D picture into 3D model, and people can see the three-dimensional version 
of Van Gogh's sunflower[11]. 

Artificial intelligence creates art and painting in a variety of forms, which can make people look at 
"painting" in a different way. In 2021, openAI launched the artificial intelligence model Dall-E, which 
spans text and image. This is a language model with 12 billion parameters. The model can generate 
corresponding images according to text description, involving two artificial intelligence fields: natural 
language processing and computer vision technology[12]. After deep learning of text data and image 
data, multimodal data is received by the model, human language is transformed into corresponding 
semantic data through natural language processing, and Dall-E "imagines" through these semantics to 
generate images that exist or do not exist in reality. This technology enables even people who do not 
understand painting to quickly make artistic painting according to the description, so that painters can 
draw even without brushes. 


of a foxsitting in a fielat sunrise 


Figure I Google's "deep dream" works (bottom left); 3D printed stereoscopic sunflower (right); 
Dall-E text generated image (top left). 
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3. Technical analysis of AI in Art Painting 


There are many kinds of artificial intelligence technology. The content applied in the field of art 
and painting needs to be carefully identified by the creator. The use of new technology will consume 
people's learning cost and time cost. When the creator applies many tools, understanding the technical 
principles is conducive to selecting appropriate tools and selecting different algorithms according to 
the content and needs of art painting. This paper analyzes the application methods of six common 
algorithms in the field of art and painting: image enhancement, image restoration, image generation, 
image fusion, text to image and style transfer. Each algorithm solves different pain points in the 
generation of art and painting. 

1) Image enhancement. This technology processes the original image through certain means, 
increases or decreases part of the information of the original image or converts part of the 
content, selectively masks or highlights a part of the image, suppresses some features that do not 
need attention, and makes the image more inclined to the characteristics to be expressed. Image 
enhancement is generally used to enhance the image to a certain extent because of light source, 
obvious noise, contrast, narrow dynamic range, insufficient definition or too high or too low 
resolution. Image enhancement often increases some information outside the image, and also causes 
partial loss of information of the image itself. When applied to art painting, image enhancement is 
mainly used to highlight the theme of art painting, increase the touch and appearance of painting 
and increase some image decoration content, which can better show the meaning that art creators 
want to promote. There is no quantitative evaluation method for the effect of image enhancement, 
which mainly depends on the psychology of the viewer and the purpose of the creator, and has strong 
subjectivity and pertinence. In practical application, a variety of image enhancement methods can be 
used to compare and test, and the focus is on the creator's perception of visual images. 

2) Image restoration. This technology uses complex algorithms to reconstruct the lost or 
damaged areas in the image. The scope of repair is mainly some defects or gaps in the image. Image 
restoration technology is a relatively mature technology at present. The algorithm can infer the lost 
information according to the color and structure information of the gap edge, and then generate the 
corresponding information content to fill the damaged area, so as to achieve the purpose of repairing 
the image. Common algorithms are: pixel based anti color neutralization; Boundary first sequential 
filling algorithm; Case based image restoration; Texture synthesis method; Image mosaic based on 
plane structure guidance; And the filling algorithm based on deep learning. In addition to the cost of 
computer performance in image restoration using deep learning, the traditional restoration algorithms 
have a relatively fast repair speed, but the repair effect and repair range have obvious disadvantages 
compared with the method of deep learning. The application of image restoration technology in art 
painting is that the creator can use the computer to fill in the remaining content without completing the 
whole painting, as well as the creator's repair or extended application of art painting, and use image 
restoration technology to restore the incomplete image scene, so as to make further artistic processing. 

3) Image generation. This technology is a hot research in recent years. It is very difficult for 
the computer to draw a picture from scratch. The computer needs to calculate each pixel, and has 


a precise algorithm design from color value to content distribution. Different from the idea that 
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the human brain sees the scene, imagines the scene and depicts the scene, the image faced by the 
computer is a pile of complex binary data. The algorithm learns the feature distribution according 
to a large number of images, so as to reconstruct a new image according to the features. One simple 
application is line drawing generation. Line drawing is the basis of art painting. In the early stage of 
painting creators, they often spend a lot of time learning. Through computer processing, they can turn 
photographic pictures or painting pictures into line drawing in a very short time, which can effectively 
help creators learn and apply line drawing. Image generation is a promising tool for art creators. 
Creators do not need to simply repeat those familiar simple paintings. Artificial intelligence can help 
them complete simple work. Computer generated images cover all aspects, from human face to human 
body, from architecture to outdoor landscape, from concrete to abstract. Creators can find inspiration 
from computer-generated images. Creators' works can provide computers with larger data sets. This 
virtuous cycle plays a great role in promoting the development of art and painting, and it is also a 
development trend in the future. 

4) Image fusion. The function of this technology is similar to stitching images, but it is not 
simply stitching in space size. To fuse several images into one, the algorithm will combine the 
images according to the same pixel value or other measurement methods. As a result of image fusion, 
multiple pictures are complementary in time-space correlation and information correlation, and 
the scene and description expressed by the pictures will be clearer, which has a full impact on the 
aesthetics of human eyes or the processing of machines. Image fusion is generally divided into several 
levels: signal level, signal fusion from the time of photographic image formation; At the pixel level, 
the hidden features are revealed through the extraction of color value, edge and texture; At the feature 
level, only the features of interest need to be fused, but a lot of information will be lost; The decision- 
making level is carried out in the content and symbolic meaning of image expression, which is more 
complex and abstract. For the application of art painting, the creator can create at a more refined level, 
so as to integrate the works at different levels. Compared with people's subjective treatment, the effect 
of integration has higher control. The creator can create in a variety of different sensory experiences 
or scene distribution, and fuse the obtained works with special images to achieve an unconventional 
painting perspective. If you are good at applying this technology, you will have a richer creative 
perspective. 

5) Text to image. Based on the image generation technology, this technology can generate 
corresponding semantic images according to the text description, such as depicting a picture and using 
natural language, such as "a cat lies on the cushion". The computer first processes the obtained text 
information in natural language to obtain the meaning to be expressed by the text: cat, cushion, lying, 
etc., and separates the objects and actions in the text into semantic elements, These semantic elements 
correspond to corresponding picture elements, such as the picture of cat corresponding to cat, and 
so on; Then create the image content according to the recognized action and scene information, and 
finally generate a logical image in the neural network model. At present, the hot technology of text 
to image technology is openAI's Dall-E. the effect and quality of generation are very excellent at 
present[13]. Dall-E can not only generate the pictures of concrete objects in reality, but also generate 
abstract and unrealistic pictures. In the creation of art painting, the technology of text to image can 
greatly facilitate the creation of creators, reduce the creation threshold of art painting, and make it 
possible for a writer to "draw" a wonderful art painting. Interdisciplinary artistic creation becomes 


116 


chinaXiv:202301.00045v1 


ChinaxXiv (ERAT! 


2022 Artistic Design, Communication and Engineering Science 


simpler. The creator can further improve the generated image, and a variety of generated effects can 
be superimposed and modified. The application of this technology will greatly expand the application 
field of art painting. 

6) Style transfer. This technology uses artificial intelligence algorithm to extract the style of one 
picture and then integrate it with another picture, so that the content of the second picture remains 
unchanged, but has the style of the first picture. Style transfer model is a focus of deep learning model 
research in recent years. Through a large number of learning of existing paintings, the computer 
can reset the new pictures into the corresponding style map in a very short time, so that works such 
as Van Gogh's starry sky and Picasso's Guernica can be presented with the same style and different 
contents[ 14]. The realization of style transfer greatly reduces the time and learning cost of simulating 
the works of different painters. Creators can use this technology to endow their works with more 
styles and create unique content. Computers do not need to spend a lot of time and energy like people, 
and have a lot of perceptual powers such as observation and imagination. They can create almost non 
repetitive works without interruption. However, the painting created by artificial intelligence does not 
give its works emotion or soul, and can not experience all kinds of perception and complex experience 
of human beings, which can only be realized by the integration of creator and artificial intelligence. 
For the application of this technology, the creator should tend to stand on the artistic style floating on 
the surface and innovate at a higher level. The style transfer technology provides the appearance of 


creation, and the creator endows it with essential connotation. 
4. Algorithmic practice of style transfer art painting generation 


The style transfer algorithm used in art painting is the current mainstream painting generation 
algorithm. The style transfer algorithm mainly uses VGG network model. VGG model plays a great 
role in promoting the research of computer vision and also plays a very important role in the field 
of art painting generation. Vgg-16 model is one of the VGG models. The model has 13 convolution 
layers, 5 pooling layers and 3 full connection layers. Dropout and L2 regularization are used for the 
first two full connection networks to prevent over fitting, and batch gradient Descent&Momentum 
are used for training optimization with cross entropy as the target loss[15]. In order to realize real- 
time and dynamic art painting, we need to build multiple images in one second, and the required 
effect can be achieved by using fast style transfer algorithm. Conventional style transfer takes up a 
lot of computing performance, and it often takes a few minutes or even hours to generate a style art 
painting. The latest style transfer algorithm in 2021 takes 10 minutes to generate a style transfer graph 
by the drafting and revision model created by Tianwei Lin et al., which is far from the demand for 
real-time dynamics[16]. For real-time and dynamic style transfer, the results of style transfer should 
be displayed when the audience can accept the waiting. Fast style transfer can meet the needs of real- 
time and dynamic. 

The fast style transfer algorithm is generally based on the pre training model of VGG. It can 
quickly convert a picture into art painting, and generate more than ten style transfer images in one 
second. It is very effective in video and real-time images. Fast style transfer integrates content and 
style information. A new loss function needs to be constructed, including content loss and style loss: 
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lrotal(@,X,p) = a K style(Q,X) + P Hoontent(P,X) 


l total (a, X P ) is the loss function of the new model, q and B are the weighted values 


of content loss and style loss, I style (a,x) is the content loss function, lcontent(P,X) 
is the style loss function. The calculation of general style loss function needs to integrate texture 


features, color features and other information. 

The overall structure of rapid style transfer is generally divided into generation network and 
loss network. The generation network can be used to generate style pictures after training. After 
the generation of style information training, the content of the pictures can be modified, so that the 
style of the pictures can be controlled at will to form style transfer Art painting. The process of fast 
style transfer includes: obtaining the pre trained VGg model parameters and using some of them for 
the new model, which saves redundant features and speeds up the speed; Then reconstruct the loss 
function, which includes content loss function and style loss function. The model loss function is 
equal to the weighted sum of the two. Finally, train according to the new loss function. Its final speed 
is faster than the general style transfer. As shown in Figure 2 and Figure 3. 
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Figure 2 Comparison between VGg model and fast style transfer loss 
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Figure 3 comparison between vgg-16 and fast style transfer processing 
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5. Works of art and painting 


Synthesize the artificial intelligence technology and style transfer algorithm mentioned in chapters 
3 and 4 to verify the actual works generated by art painting. Two different technical routes are used to 
generate dynamic line drawing and real-time style transfer painting. Dynamic line drawing generation 
can generate dynamic line drawing pictures in real time on video and image materials. The efficiency 
is unmatched by conventional art painting. It involves technologies such as image enhancement and 
image restoration, showing the preliminary application mode of cross-border integration of artificial 
intelligence and art painting. Real time style transfer painting uses convolution neural network for 
training and generation. The speed of generating works with characteristic style is much faster than 
that of human painters. It is a deep application way of cross-border integration of artificial intelligence 
and art painting. 


5.1 Real time dynamic line drawing image generation 


In the research, the corresponding artificial intelligence algorithm is used to convert pictures, 
videos and real-time images into line drawing pictures. During the experiment, we use the computer 
as the program running platform, the camera captures the real-time picture, and the display shows 
the picture after real-time processing. In the algorithm design, firstly, the original image is processed 
into gray image. The original image has three color channels of red, green and blue, while the gray 
image has only one color channel. The displayed black-and-white image is easier to process and 
reverse image; Gaussian filter and mean filter are used to denoise the original image to erase the noise 
and uncoordinated changes in the image and make the image smoother; Then, the gradient size and 
direction of each pixel in the image are calculated by using the first-order partial derivative erasure 
difference, the calculated gradient is suppressed by non maximum value, the interference signal at 
the image edge is eliminated, the edge blur effect caused by gradient calculation is reduced, and a 
high threshold and low threshold are set to filter the size of edge gradient, which can eliminate the 
invalid gradient value caused by noise and color change; Finally, check the connected domain pixels 
connected by each edge gradient after filtering, and select the real edge. The algorithm can not only 
achieve good line drawing generation effect, but also has very fast processing speed. According to the 
experiment, it can generate 22 line drawings per second, which meets the requirements of real-time 


dynamic images acceptable to human eyes. As shown in Figure 4. 
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Figure 4 flow chart of real-time dynamic line drawing generation algorithm 
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5.2 Real time dynamic style transfer art painting generation 


The work generation uses the fast style transfer algorithm to learn the 43 paintings of Marlene 
Dumas, and then carries out the style transfer and painting processing on the real-time image, so 
as to display the style of different scenes and character images in real time under the condition of 
continuous dynamic input. Most of Marlene Dumas's works focus on the changes of human face, so 
the face recognition model is used in the real-time image. The model can quickly identify and mark 
68 feature points of human face, cut the face image in each frame according to these feature points, 
extract the face in each frame, and then process the face photo into the form of line drawing, The 
effect of style transfer with line drawing image is more appropriate in Marlene Dumas's works. His 
works prefer simple scenes and pay more attention to the contour change of human face. The effect 
of style transfer with line drawing is closer to that of the original painting. The real-time dynamic 
style transfer requires more computer performance, and the model needs to be trained for a long time. 
After the training, the trained model is used for effect verification. Under the same configuration, the 
generation of 10 art paintings per second can be completed to meet the basic dynamic art painting 
effect. As shown in Figure 5. 
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Figure 5 flow chart of real-time dynamic fast style transfer generation algorithm 


6. Conclusion 


Artificial intelligence art creation is inseparable from a large amount of data. It needs to 
be adjusted and modified by the creator in order to create excellent works. At present, artificial 
intelligence art can not be perfectly presented in all as-pects of artistic creation. The future 
development trend will be that creators are good at applying artificial intelligence technology to 
develop more diverse creative methods. The missing part of this article is that it fails to fully show all 
the creative techniques and presentation methods of artificial intelligence art painting in the article, 
and our real-time dynamic style transfer art painting can also achieve better results according to the 
improvement of artificial intelligence technology. I hope relevant researchers can further explore 
artificial intelligence art painting and strive for the perfect combination of technology and art. 
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